Completeness Results for Lifted Variable Elimination: Appendix

In this document, we present proofs for Theorem 2 and 3 (given in the paper), and provide more explanation for the empirical evaluation. Further, we present a procedure for tramsforming weighted model counting (WMC) models to parfactor models. 1 PROOF OF THEOREM 2 Let us first recall the theorem. Theorem 2 C-FOVE is a complete domain-lifted algorithm for the class of models in which each atom has at most 1 logvar. Proof sketch. The proof builds on the proof of Theorem 2 (given in the paper). Note that the approach used in Steps 2 and 3 of the proof of Theorem 2 is also applicable here. The operations in Step 2, which together eliminate the 1-logvar atoms, do not depend on the total number of logvars in the parfactors. Using this approach, we can eliminate all the 1-logvar atoms in any model whose atoms contain at most one logvar. The resulting model can be solved as in Step 3. As was shown in the proof of Theorem 2, the time-complexity of these steps is polynomial in the domain size. The inference procedure is thus domain-lifted. 2 PROOF OF THEOREM 3 Let us first recall the theorem. Theorem 3 Lifted sum-out with the group inversion operator is sound, i.e., it is equivalent to summing out the randvars on the ground level. Appearing in Proceedings of the 16 International Conference on Artificial Intelligence and Statistics (AISTATS) 2013, Scottsdale, AZ, USA. Volume 31 of JMLR: W&CP 31. Copyright 2013 by the authors. We prove the theorem by showing that the corresponding ground operations are both independent and isomorphic. Independence. We require the following definition. Given a set of factors F and set of randvars R, we call a subset of factors F ′ ⊆ F mutually closed with respect to a group of randvars R′ ⊆ R, if (i) no factor in F \ F ′ contains a randvar r′ ∈ R′, (ii) no randvar in R \ R′ appears in a factor f ′ ∈ F ′, and (iii) each randvar r′ ∈ R′ appears in some factor f ′ ∈ F ′. Now, we show that we can form mutually closed sets of randvars and factors in R = RV (Ai|C) and F = gr(g) by partitioning them into sets in which all elements are permutations of each other (can be derived from one another by a permutation of constants). The set of permutations that defines the partitioning is the minimal permutation group [Λ]. Given a set of permutations Λ on X, we define two substitutions θ1, θ2 to be in the relation ∼Λ iff λ(θ1) = θ2 for some λ ∈ Λ. Using this relation we can define a partitioning of a set of substitutions Θ as ΘΛ, where θ and θ′ are in the same group if and only if θ ∼Λ θ′. As shown in steps 1 and 2 of the operator, for any two factors gθ and gθ′ that share a randvar from the set RV (Ai), we have θ = λ(θ ′), for some λ ∈ [Λ]. Thus for any Θi ∈ Θ[Λ], the set of factors Fi = {gθ|θ ∈ Θi} are mutually closed w.r.t. the set of randvars Ri = {Aiθ|θ ∈ Θi}. This shows that we can divide the problem of summing out RV (Ai) from gr(g) into independent problems of summing out each set of randvars Ri from the set of factors Fi. Isomorphism. We show that the sum-out problems are also isomorphic, by a mapping between the ground substitutions that produce ground factors in each group. To show the isomorphism between groups of gr(g), we note that each group is formed from the factors {gθ|θ ∈ Θi}, where Θi is a group in Θ[Λ]. The one-toone mapping between the factors can thus be established by a one-to-one mapping between the constants of the grounding substitutions in different groups Θi Completeness Results for Lifted Variable Elimination: Appendix and Θj . This is done by starting from an arbitrary pair of substitutions θi ∈ Θi and θj ∈ Θj and mapping the constants that are assigned to the same logvar to each other. It follows then that each substitution θ′ i ∈ Θi such that λ(θi) = θ′ i is mapped to exactly one substitution θ′ j ∈ Θj such that λ(θj) = θ′ j . As such the set of factors (and the set of randvars) are isomorphic up to a renaming of the constants in each group. This shows that the sum-out problems in different groups are independent and isomorphic. Hence, it is correct to replace them by a single lifted operation, i.e. to solve one instance of the problem for a representative group and generalize the result for all, as is performed in lifted sum-out by the group inversion operator. 3 EXPLANATION ABOUT THE EMPIRICAL EVALUATION In this section we show how C-FOVE solves each of the models used in our empirical evaluation, and compare the complexity of inference in each model. 3.1 The friends and smokers model This model consists of the following two parfactors (in normal form): g1 = φ1(S(X), F (X,Y ), S(Y ))|X 6= Y g2 = φ2(F (X,Y ), F (Y,X))|X 6= Y We first eliminate the 2-logvar F atoms, as follows. We multiply g1 and g2 to compute the product g = φ(S(X), F (X,Y ), F (Y,X), S(Y ))|X 6= Y Then we eliminate the F atoms by group-inversion, which results in the parfactor g′ = φ′(S(X), S(Y ))|X 6= Y Next, we eliminate the 1-logvar S atoms as follows. By just-different counting conversion, we rewrite g′ as g′′ = φ(#X [S(X)]) We then eliminate the S randvars by summing-out the counting formula γ = #X [S(X)] from g ′′. The result is a potential with no arguments (a constant). This concludes inference. Complexity. The most expensive step here, is the elimination of the counting formula #X [S(X)], whose range size is O(n), with n the domain size of the logvars. As such the whole process runs in time linear in the domain size. 3.2 The collective classification model Below we abbreviate Link to L, and Class to C. The model consists of the following parfactors: ∀i, j ∈ {1, 2} : gij = φij(Ci(P1), L(P1, P2), Cj(P2)) |P1 6= P2 g2 = φ2(L(P1, P2), L(P2, P1))|P1 6= P2 Inference in this model follows the same steps as the friends and smokers model. We first eliminate the 2-logvar L atoms, as follows. We multiply all the 5 parfactors g2, g11, g12, g21 and g22 to compute the product g: φ(C1(P1), C2(P1), L(P1, P2), L(P2, P1), C1(P2), C2(P2)) |P1 6= P2 Then we eliminate the L atoms by group-inversion, which results in the parfactor g′ = φ(C1(P1), C2(P1), C1(P2), C2(P2))|P1 6= P2 Next, we eliminate the 1-logvar atoms C1, C2 as follows. By joint conversion on C1 and C2, we rewrite each of their occurrences as a joint atom J12 φ(J12(P1), J12(P1), J12(P2), J12(P2))|P1 6= P2, which after a simplification of recurring atoms becomes: g′ = φ(J12(P1), J12(P2)) |P1 6= P2, Then, by just-different counting conversion, we rewrite g′ as g′′ = φ′′′( #P [ J12(P ) ] ) Finally, we eliminate the C1 and C2 randvars by summing-out the counting formula #P [ J12(P ) ] from g′′. The result is a potential with no arguments (a constant). This concludes inference. Complexity. The most expensive step in this process is the elimination of the counting formula #P [ J12(P ) ], whose range size is O(n|range(J12)|−1), with n the domain size of the logvars. Note that here |range(J12)| = 4, since range(J12) = range(C1) × range(C2). Thus the whole process runs in time O(n), i.e., complexity of inference is cubic in the domain size.